1. Field of the Invention
The present invention relates to an image processing device capable of removing noise from an image, and more particularly to an image processing device that enables to remove noise from a wide dynamic range image obtained by an image sensing apparatus capable of performing a wide dynamic range imaging, an image processing method, and an image sensing apparatus using the image processing method.
2. Description of the Related Art
An image sensing apparatus such as a digital camera is provided with a predetermined image sensor to capture an image by an imaging operation of the image sensor. Generally, an image i.e. an image signal captured by an image sensor includes a noise component resulting from e.g. a dark current inherent to the image sensor. In response to a recent demand for a high-quality image, it is required to finely remove the noise component.
As an example of the conventional noise removal methods, there is known a method, as shown in FIG. 25, comprising: isolating a high-frequency component 922 including a noise component, or as a noise component by subtracting a low-frequency component 921 which does not include the noise component and is extracted by an LPF (low-pass filter) processing, from the captured image; and performing a noise removal processing i.e. a noise component removal processing such as a coring processing with respect to the high-frequency component 922 to remove the noise component. It is highly likely that the high-frequency component 922 which is isolated as the frequency component including noise may include a frequency component as a real component of the image i.e. a high-frequency component 9222, in other words, frequency noise is superimposed. The high-frequency component 9222 is a component whose frequency is higher than that of the low-frequency component 921, but is lower than that of a high-frequency component 9221, or a high-frequency component 9223 substantially equal to the high-frequency component 9221 in frequency level. If the noise removal is performed by the conventional method, the real image component may also be removed with the noise component.
FIG. 26 is a diagram showing the high-frequency components 9221, 9222, and 9223 on the same drawing for convenience of explanation. As shown in FIG. 26, the high-frequency components 9221 and 9223 are noise components, and the high-frequency component 9222 is an edge component, which is a real image component. In the case where a noise removal amount is set, and the noise component in the range of the noise removal amount is removed, it is possible to remove the entirety of the right-side-located high-frequency component 9223 without any processing. However, concerning the left-side-located high-frequency components 9221 and 9222, it is impossible to remove the noise component at the portions indicated by e.g. the reference numerals 923, 924, and 925, because these portions 923, 924, and 925 are out of the range of the noise removal amount. Further, if the portion of the high-frequency component 9221 indicated by e.g. the reference numeral 926 is attempted to be removed, because the portion 926 is in the range of the noise removal amount, an edge portion indicated by the reference numeral 927 i.e. the high-frequency component 9222 is also removed.
In view of the above, as a method for removing a noise component while preserving an edge component, there is disclosed a technique in e.g. Japanese Unexamined Patent Publication No. 2001-298621 (D1). D1 discloses a method comprising: generating a low-frequency component whose edge component is preserved by an epsilon filter processing using ε filters arranged in series; and performing a coring processing with respect to a high-frequency component generated by subtracting the low frequency component from an original image for noise removal. The epsilon filter processing is not a processing to be executed by hierarchical steps, which will be described later. With use of the technique, the low-frequency component can be extracted from the original image in such a manner that the edge component is included in the low-frequency component. In other words, the edge component is not included in the high-frequency component to be removed as the noise component. Accordingly, the edge component is preserved without being affected by the noise removal processing. However, in this technique, if superimposed noise that the noise component is superimposed over the real image component is included, it is impossible to exclusively remove the noise component from the superimposed noise.
In light of the above drawback, e.g. Japanese Unexamined Patent Publication No. 2000-134625 (D2) discloses a method comprising: dividing a high-frequency component including a superimposed noise component into plural frequency components; and isolating the noise component from the real image component for noise removal. Specifically, this method comprises: in performing a frequency band division processing of dividing an input image into plural frequency components i.e. frequency band components, referring to a high frequency component i.e. a middle frequency component generated by a succeeding division processing with respect to a high-frequency component to be removed as a noise component; and changing the currently generated high-frequency component based on a normalization coefficient to be used in edge detection, which has been calculated based on a maximal value of the succeedingly generated high-frequency component. The normalization coefficient is a coefficient having a property that the coefficient is set to a large value if the edge component is detected, and is set to a small value if the edge component is not detected. The edge component is preserved by multiplying the normalization coefficient with the high-frequency component to be removed as the noise component, and the noise component other than the edge component is removed. Thus, the noise component is isolated from the real image component in the high-frequency component by the frequency band division processing in calculating the normalization coefficient. Thus, the noise component is exclusively removed, even if the superimposed noise is included.
In recent years, as the high quality image is demanded in the technical field of image sensing apparatuses such as digital cameras, there is a task of increasing a luminance range i.e. a dynamic range of a subject to be handled by an image sensor. Concerning the technique of increasing the dynamic range, there are known e.g. an image sensor using logarithmic compression i.e. a logarithmic sensor, and a linear-logarithmic sensor. The logarithmic sensor is constructed in such a manner that an electric signal commensurate with an incident light amount is logarithmically transformed and outputted. The linear-logarithmic sensor has a photoelectric conversion characteristic including a linear characteristic that an electric signal is linearly transformed and outputted in a low luminance area, and a logarithmic characteristic that the electric signal is logarithmically transformed and outputted in a high luminance area. An image which is captured by the linear-logarithmic sensor, and has the linear characteristic and the logarithmic characteristic of the linear-logarithmic sensor is called a “linear-logarithmic image”. With use of these image sensors, a naturally logarithmically transformed output is obtained with respect to the incident light amount. Accordingly, these image sensors are advantageous in capturing an image having a wider dynamic range, by a one-time exposure operation, as compared with an image sensor having a photoelectric conversion characteristic merely with a linear characteristic.
In the current technology, whereas a wide dynamic range is secured in an imaging system, as an imaging device such as the linear-logarithmic sensor has been developed, a wide dynamic range is not secured in a display system i.e. an image display device such as a monitor, as compared with the imaging system. Even if a wide dynamic range is secured in the imaging system, the effect of the wide dynamic range cannot be satisfactorily exhibited on the display system having a relatively narrow dynamic range, as compared with the imaging system. In other words, a captured image with a wide dynamic range is compressed in conformity with the dynamic range of the display system such as the monitor. Accordingly, a resultantly obtained image has a low contrast, which obstructs proper reproduction i.e. display of the captured image.
In view of the above, it is required to perform a gradation conversion processing i.e. a contrast emphasis processing such as dynamic range compression processing of e.g. extracting an illumination component and a reflectance component from a captured image, and compressing the illumination component so that the captured image with a wide dynamic range is displayed in the dynamic range of the display system. In the specification and the claims, an image with a wide dynamic range which requires a gradation conversion processing such as a dynamic range compression processing to reproduce the image on a display system having a narrow dynamic range, and consequently requires a gradation conversion processing such as a dynamic range compression processing with a larger compression rate e.g. a larger amplitude rate or a larger gradation conversion rate, as compared with an ordinary compression processing, is referred to as a “wide dynamic range image”.
In the wide dynamic range image, the noise component as well as the real image component is greatly amplified by the gradation conversion processing. As a result, the noise is emphasized, as compared with an image having an ordinary dynamic range. (not a wide dynamic range) i.e. an ordinary dynamic range image or a standard dynamic range image. The techniques disclosed in D1 and D2 involve the following drawbacks, which are not involved in processing the ordinary dynamic range image by the conventional noise removal processing.
Specifically, in the technique disclosed in D1, it is impossible to isolate the noise component from the superimposed noise in a condition that the noise component is superimposed on the real image component, and accordingly, it is impossible to exclusively remove the noise component from the superimposed noise. However, as far as the ordinary dynamic range image is processed, a serious drawback is not involved, because the noise component with respect to the real image component is negligibly small. However, in the case where the wide dynamic range image is processed, it is highly likely that the noise component with respect to the real image component may be significantly large. As a result, the noise component in the image may be intolerably large.
In the technique disclosed in D2, as mentioned above, the succeedingly generated high-frequency component in the frequency band division processing is referred to for calculating the normalization coefficient to be used in edge detection. The high-frequency component is a component to be isolated in extracting the noise component in the noise removal processing. Therefore, it is highly likely that the noise component may remain in the high-frequency component. The edge component is preserved by referring to the high-frequency component. Accordingly, the noise component may be erroneously detected and preserved as the edge component. If such a phenomenon occurs, the noise component may adversely affect the image because it is highly likely that the noise component is non-negligibly large in the wide dynamic range image.